Interactivity with a mixed reality

ABSTRACT

Methods of interacting with a mixed reality are presented. A mobile device captures an image of a real-world object where the image has content information that can be used to control a mixed reality object through an offered command set. The mixed reality object can be real, virtual, or a mixture of both real and virtual.

This application is a CIP of PCT Application Serial No. PCT/U.S.06/33811filed Aug. 29, 2006, which claims priority to U.S. ProvisionalApplication No. 60/712,590 filed Aug. 29, 2005. These and all otherextrinsic references are incorporated herein by reference in theirentirety. Where a definition or use of a term in an incorporatedreference is inconsistent or contrary to the definition of that termprovided herein, the definition of that term provided herein applies andthe definition of that term in the reference does not apply.

FIELD OF THE INVENTION

The field of the invention is mixed reality technologies.

BACKGROUND

Virtual reality systems are having an ever increasing impact on ourdaily lives. Virtual world games, for example, continue to dominate thegaming industry and are migrating to mobile devices of all sortsincluding cell phones. Just as virtual worlds are impacting our reallives, the real world will have an ever growing impact on our virtuallives. The distinction between virtual worlds and the real world willcontinue to become ever more blurry resulting in an ever present mixedreality. People living with the various mixed realities will interactwith mixed reality objects whether they are real, virtual, or a mix ofboth. To date there are no easy, intuitive methods for a person tocommand or control mixed reality objects.

Google Maps™ provides a method for people to interact with a mixedreality where real-world satellite images are superimposed over computergenerated images showing location or road names. Although Google Maps isuseful for interacting with maps and for virtual travel, it does notaddress command or control of mixed reality objects.

U.S. Pat. No. 6,502,000 to Arnold et al. offers a system for controllingreal-world devices through a communication link between a controlleddevice and a controlling device but does not address the need forinteracting with virtual objects of a mixed reality. The Arnold approachis useful for closed ecosystems where all devices use a common controlprotocol. However, the approach does not lend itself to a universal,heterogeneous environment where mixed reality objects employ multiplecommand or control protocols.

The mixed reality system described by U.S. Pat. No. 6,604,049 to Yokotateaches that a person can interact with a mixed reality by accessing anddisplaying information about a real-world object on a virtual realitydisplay device worn by the person. A camera captures images ofsurrounding objects and sends the image data to a remote server forprocessing. The server then returns information associated with theobjects for display on the user's display screen. Although useful forobtaining information about real-world objects, the system does notteach how to command or control mixed reality objects. Additionally, itis unlikely individuals will purchase or wear the contemplated virtualreality system as opposed to using a system the individual already owns,possibly a cell phone or PDA.

U.S. Publication No. 2002/0090985 to Tochner et al. further advancesmixed reality interactions by teaching that a player of a virtual worldgame can interact with the real-world by establishing a correspondencebetween virtual objects and real-world objects. Still, Tochner does notprovide for the command or control of mixed reality objects.

The applicant's patent U.S. Pat. No. 7,016,532 and patent applicationsU.S. Publication Nos. 2004/0208372, 2006/0002607, 2006/0110034, and2006/0181605 teach that one can use a mobile device to capture imagedata associated with a real-world object to access information about thereal-world object. The image data is used to identify network addresseswhere information associated with the real-world objects can beobtained. The network addresses can also be used to perform an actionrelating to the real-world objects. Although the applicant's previousfilings teach performing an action associated with a returned networkaddress and controlling real-world objects, those filings do not teachhow to command or control mixed reality objects.

What has not yet been appreciated is that a mobile device can provide aninterface to command or control mixed reality objects by capturing animage of a real-world object. Thus, there is still a need for systemsand methods for interacting with mixed realities.

SUMMARY OF THE INVENTION

The present invention provides apparatus, systems and methods in which amobile device can be used to command or control objects within mixedrealities.

One can interact with a mixed reality by capturing an image of areal-world object using a mobile device where content from the image isused to derive a command set for a mixed reality object that isaccessible by the mobile device. The mixed reality object can include avirtual object in a virtual world, a second real-world object, or anobject having presence within a virtual world and the real world.Command sets are contemplated to have commands including changingchannels, turning a device on or off, or inputting data.

As used herein, “real-world objects” are those that have mass and extentin the physical world, and “virtual objects” are those that do not havemass and extent in the physical world. The term “mixed reality” means areality having at least one real-world object and at least one virtualobject, which a user of the mixed reality space can perceive asinteracting with one another in some manner.

Mixed realities cover the spectrum from slight modifications of thephysical world where virtual elements are superimposed on the realworld, to fully virtual worlds where a real-world object has acorresponding presence in the virtual world. In general, a mixed realityrequires support from an electronic interface, possibly a cell phone orPDA, to view or access the mixed reality. The real-world itself is not amixed reality because there are no virtual elements. Many computer gamesper se are virtual realities, but they are not mixed-realities becausethey are self contained and do not share real-world elements. Othercomputer games supporting virtual worlds can represent a mixed-realitywhen a real-world object has a corresponding presence within the gameworld including trading cards, money, avatars, or other representationsof real-world elements. An example of a mixed reality weighted heavilytoward the virtual end of the spectrum includes, Linden Lab's SecondLife® where real-world dollars correspond to money in Second Life.

A “mixed reality object” is an object within a mixed reality capable ofinteracting with other objects in the mixed reality. A physicaltelevision per se is a real-world object and not a mixed reality object,even though it can be used to display virtual or mixed reality content.The physical television set can be part of a mixed reality when it has acorresponding presence in a mixed reality space. For example, if a userturns off the physical television, the corresponding presence in themixed reality space could be shown as having been turned off.

Various objects, features, aspects and advantages of the inventivesubject matter will become more apparent from the following detaileddescription of preferred embodiments, along with the accompanyingdrawings in which like numerals represent like components.

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1A is a schematic of a user using a mobile device to capture imagedata of a real-world object.

FIG. 1B is a schematic of the mobile device of FIG. 1, displaying animage of a mixed reality world that includes a portion of the imagedata, and command sets associated with one or more objects in the mixedreality objects.

FIG. 2 is a schematic of methods for interacting with a mixed reality.

DETAILED DESCRIPTION

Mixed Reality Environment

In FIG. 1A, user 100 interacts with a mixed reality environment thatincludes real-world object 120.

In a preferred embodiment, user 100 uses mobile device 110 as a commandor control interface for mixed reality objects that are local to mobiledevice 110. Although the terms “local” and “locality” are contextdependent, as used herein two objects are considered “local” to eachother when they are physically or virtually near each other. Physicallylocal objects are generally within about 100 meters of each other. Morepreferably, objects are considered local when they are within 10 metersof each other. It is also contemplated that a user could interact withobjects that are more than 100 meters from mobile device 110.

Virtual objects are preferably within visual range of each other in themixed reality. It is also contemplated that virtual objects could behidden from view, yet remain in visual range. For example, a virtual petthat can only be seen using mobile device 110 might be hiding behind areal-world car, yet the virtual pet is still virtually local to mobiledevice 110.

Although a preferred embodiment calls for mobile device 110 to commandor control local mixed reality objects, the techniques disclosed canalso be applied to remote mixed reality objects which also fall withinthe scope of the inventive subject matter.

Mobile device 110 comprises display 113, camera 115, and networkconnection 119. Through the use of these elements, mobile device 110generally becomes an interface portal into the mixed reality utilizingsupport from software running at least partially on external computersystems. One should appreciate that mobile device 110 or its abstractedinterface, can be considered an object within the mixed reality.

Display 113 is contemplated to include a GUI (Graphical User Interface)that can display image data or user interfaces. User 100 interacts withthe mixed reality through the interface rendered by display 113.Typically, display 113 comprises an LCD (Liquid Crystal Display) as usedin computers, cell phones, PDAs, or other similar devices. Additionallymobile device 110 preferably runs a browser application or other GUI forrendering graphics or command interfaces on display 113. In someembodiments, display 113 renders the real-world object along withvirtual objects that are objects in the mixed reality. It is alsocontemplated that display 113 could comprise non-visual displays for useby other senses. For example, a blind person could use a tactile displayto interact with a mixed reality as opposed to a visual-based display ora deaf person could use an auditory interface system to interact with amixed reality.

Camera 115 captures image data of real-world object 120 by capturingstill images, video images, streamed images, or other image data. CCDcameras as typically used in hand-held device are a contemplated form ofcamera. Any camera capable of electronically capturing image data wouldalso be suitable. The image data can be captured through manuallysnapping a single still frame, recording audio or video, or simply byhaving the camera stream real-time images. In some embodiments, camera115 has advantageous features including zoom, infrared sensitivity, orultraviolet sensitivity which can increase the amount of informationobtained relating to real-world object 120.

Image data of real-world object 120 captured by camera 115 combines withadditional data obtained over network connection 119, possibly fromcomputer 130 or real-world object 120, to create a interactive view ofthe mixed reality. The additional data can also be obtained by mobiledevice 110 and can include time, temperature, location (e.g. latitude orlongitude), or other information.

Network connection 119 preferably utilizes standard Internet relatedprotocols including TCP, UDP, IP, HTTP, wired protocols, wirelessprotocols, or other communication protocols; although all communicationprotocols are contemplated. Mobile device 110 uses network connection119 to communicate with other devices or mixed reality objects externalto mobile device 110 including real-world object 120 (if it is soenabled) or computer 130.

Mobile device 110 can also comprise optional sensor 111. Sensor 111acquires additional data about mobile device 110 or real-world object120 for use in interacting with the mixed reality. Contemplated sensorsinclude audio sensors, accelerometers, orientation sensors, GPS (GlobalPositioning System) or other physical location sensors, or other sensorsbeyond camera 115. Additional sensor 111 provides information that canbe used by software to further identify attributes associated withreal-world object 120 through the location or the orientation of mobiledevice 110. Accelerometers provide motion information that can be usedto determine how the mobile device moves relative to real-world object120 which can be used to alter the visual display of virtual objects orcan be used as command inputs to control objects. For example, U.S. Pat.No. 7,271,795 to Bradski describes a suitable method for using sensorinput to alter a visual display when interacting with a virtual space.

Mobile device 110 is preferably a hand-held device including a cellphone, a PDA, a remote control, a hand-held computer, or other similarportable device. It is also contemplated that mobile device 110 couldinclude larger, non-hand-held devices. For example, mobile device 110could be a vehicle, potentially a police car, equipped with a camera, adisplay, or a network connection. A police car so equipped can aidpolice officers by automatically viewing a license plate, for example,and providing the office with a list of commands relating to the car,the driver, the driver's record, or other real or virtual objects.

Real-world object 120 can be any object in the real world that can beimaged whether it is a machine, a person, a building, a plant, a set ofobjects, or other physical items. In embodiments where real-world object120 is a machine, it is contemplated that real-world object 120 alsocommunicates with computer 130 to support interactions with the mixedreality.

One should note that real-world object 120 does not necessarily have tobe electronic or mechanical to be commanded or controlled within a mixedreality. Real-world object 120 could be co-existent with a virtualobject within a virtual world where the virtual representation ofreal-world object 120 can be commanded or controlled. Consider, forexample, a virtual game where a real world fire hydrant corresponds to avirtual fire hydrant in the virtual game world. Although a real playercannot operate the real fire hydrant, the player could capture imagedata of the real fire hydrant and then command or control thecorresponding virtual fire hydrant as part of the game.

Real-world object 120 is also contemplated to include multiple partsthat can be controlled individually or collectively through an image ofat least a portion of the object. For example, a stereo systemrepresents a multi-part object that could include a CD player, speakers,a tuner, a graphical analyzer, or other parts. When image data of thestereo is captured, one or more virtual interfaces, each having acommand set, can be superimposed over the image of the stereo or itsindividual parts. The act of combining the virtual interface with theimage data from the real-world essentially forms a mixed realityrepresentation of the stereo.

Computer 130, external to mobile device 110, runs software for analyzingimage data to recognize attributes associated with real-world object120. The preferred embodiment employs techniques similar to thosedescribed in co-owned U.S. Pat. No. 7,016,532 where a remote serviceidentifies network addresses associated with a target object and returnsthe network address to a mobile device.

The software running on computer 130 communicates with software onmobile device 110 and, potentially, with software running on real-worldobject 120 to support the structure of the mixed reality environment. Itis also contemplated that computer 130 can be incorporated intoreal-world object 120.

Computer 130 can be one of any of a full spectrum of computer resources.At the real-world end of the mixed reality spectrum, computer 130 cancomprise a web server or database that supplies command or controlinformation to mobile device 110. At the other end of the mixed realityspectrum where the virtual aspects dominate the mixed reality, computer130 can include a virtual world server or a game server that supplyvirtual world environment information for use by mobile device 110.

Mixed Reality Interface

In FIG. 1B, mobile device 110 displays a mixed reality interface ondisplay 113. The mixed reality interface can include an image ofreal-world object 120 having command set 125, or virtual object 150having command set 155.

Mobile device 110 uses camera 115 to capture image data of real-worldobject 120 and sends at least a portion of the image data, attributes,or other optional information from sensor 111 to external computer 130where attributes associated with real-world objects 120 are identified.

As previously mentioned, computer 130 uses the attributes to identifyone or more network addresses or other commands using techniques similarto those described in U.S. Pat. No. 7,016,532. The techniques areextended to include any virtual objects within the mixed reality thatare accessible to mobile device 110. Computer 130 returns informationassociated with one or more of the mixed reality objects (either real orvirtual) to mobile device 110 including data necessary for renderingvirtual object on display 113.

Consider a case where the local object is a real object. Attributes fromthe image data associated with real-world object 120 are identified andused by the software to offer command set 125 to control real-worldobject 120 which can be physically local to mobile device 110. As shownin FIG. 1B, real-world object 120 can include a vending machine. Oneshould note that term “offer” is used euphemistically to mean expose orotherwise provide access to command set. A user, mobile device 110, orother entity interacts with the vending machine by interfacing withcommand set 125 listing one or more vending machine commands.Contemplated commands include purchasing an item or possibly checkingthe inventory of the machine. Other contemplated commands include thosethat might require proper authentication including turning the vendingmachine on or off. One should appreciate that the commands are notlimited to those described within this document and that all possiblemixed reality objects commands are contemplated.

Also consider the case where a mixed reality object is a virtual object.Again, attributes derived from the image data or optional sensor dataare used to by the software to determine if there are virtual objectslocal to mobile device 110 within the mixed reality. The software alsouses the attributes or optional sensor data to offer command set 155.Command set 155 offers one or more commands to control virtual object150 which, as shown in FIG. 1B, can include a virtual pet. Examplecommands for virtual object 150 include turning the virtual object on oroff, instructing the object to move, upgrading the object, or any othercommand that can alter the operation of virtual object 150. One shouldalso appreciate that virtual objects remote to mobile device 110 canalso be controlled.

In some embodiments, the software on computer 130 runs in conjunctionwith software on mobile device 110 to ensure virtual objects aredisplayed in proper relation to real objects. For example, virtualobject 150 can be rendered in a manner where it is partially hidden byreal-world object 120 even though the virtual object does not exist inthe real world and is not part of the original image data.

One should appreciate that in a mixed reality, virtual objects and realobjects co-exist on equal footing as if they are both natural to themixed reality. Therefore, the manner in which one interfaces to avirtual object of the mixed reality using mobile device 110 issubstantially identical to the manner in which one interfaces to a realobject. Even though the infrastructure for a mixed reality system mighttreat real and virtual objects differently; the user interface of eachtype of object is preferably similar so that a user experiences bothtypes of objects as being natural to the mixed reality.

Command sets 125 and 155 preferably include one or more commands. Acommand set can be displayed using any suitable types of command promptsincluding menus, radio buttons, simple list of one or more commands,drop-down lists, command line prompt, icons that open to reveal thecommand set, graphical controls, or other types of command offerings.Contemplated command input mechanism to access a command include callingan API, handset motion, stylus, rocker buttons, voice commands, or otherinput methods.

It is also contemplated that a command set can be hidden from the user,but yet is still accessible through mobile device 110. Consider theexample where real-world object 120 is a vending machine. Command set125 for the vending machine might not be visible to the user whilemobile device 110 is still aware of command set 125. A user could accesscommand set 125 by interacting directly with the image of the vendingmachine, including “clicking” on the image of one of the vending machinebuttons. Alternatively, mobile device 110 could access command set 125automatically using one or more API calls without direct interactionwith the user. Furthermore it is also contemplated that other remotedevices (e.g. computer 130) could also access commend set 125 to controlthe mixed reality object.

One should also appreciate that the command bubbles displaying commandsets 125 and 155 are objects of the mixed reality. It is, therefore,contemplated that as a user pans, tilts, or otherwise moves mobiledevice 110, the command set can also optionally move within display 113to stay in proper relation to object within the mixed reality view.

In a preferred embodiment, the commands in a command set comprise one ormore network addresses obtained from computer 130. The network addresscan include URLs, URIs, remote procedure calls, email addresses, IPaddresses, web service APIs, or other network callable commands oraddresses. When a user selects a command from the command set,preferably mobile device 110 executes the command by issuing a call tothe corresponding network address. In some embodiments, the networkaddress points to a function on computer 130, possibly where computer130 is running as a web server or a virtual world server. In otherembodiments, the network address points to an accessible function onreal-world object 120, or other mixed reality object accessible bymobile device 120. Although the preferred embodiment employs webservices, it is also contemplated that other types of command servicescan also be used including email, FTP, Telnet, proprietary services, orother services capable of executing a command. It is also contemplatedthat a command associated with a virtual object can be at leastpartially executed internal to mobile device 110 as opposed to executingon a remote computer system.

Mixed Reality Interactions

In FIG. 2, method 200 describes how one can interact with a mixedreality.

At step 210 a mobile device is preferably used to electronically captureimage data of a real-world object. The real-world object is consideredto have mass and extent and actually exist in the physical world. Areal-world object can include anything that can be photographedincluding a person, a place, or a thing. Image data is contemplated tocomprise at least a still image at step 212 or a video image at step214. One should note that a projected image is not considered areal-world object because is lacks mass, but is rather considered avirtual object. However, a photograph, a mural, a poster, a magazineadvertisement or other physically rendered image is considered to be areal-world object. Although a preferred embodiment captures image dataof a real-world object, it is also contemplated that one could captureimage data of a virtual object without significantly departing from theinventive subject matter.

At step 220 attributes of the real-world object are identified from theimage data. In some embodiments, the mobile device comprises software toderive the attribute, however, in preferred embodiments the image datais transmitted to a remote computer which identifies the attributes. Itis also contemplated that any combination of software applications,local or remote to the mobile device, can be used to derive theattributes.

Attributes include metadata associated with the content of the imagedata, and relate to objects in the image. Attributes are used toidentify or to derive a command set associated with various mixedreality objects accessible by a mobile device. At step 222, preferredattributes include a name, a class, or an environmental characteristicassociated with a real-world object.

Names are useful when offering a specific command set for a targetobject. For example, if a user captures an image of a Xerox® 4150PSprinter, then the name “Xerox 4150PS” can be used to access specific4150PS command functions and provide them to the user.

Classes represent broad categories of objects. To continue the previousprinter example, an appropriate class for the Xerox printer is“printer”. Classifying objects provides for offering generic commandsthat are common across many types of objects or devices. A common set ofcommands for many appliances include “on” or “off” commands. In apreferred embodiment, classes are organized hierarchically as in aclassic object oriented programming methodology.

Environmental characteristic are attributes that indicate theenvironmental condition in which the real-world object exists. Examplesinclude weather conditions, time of day, geographic location, or otherenvironmental information. Environmental characteristics areadvantageously used to adjust command sets to meet the potential needsof a user or alter the behavior of the mixed reality. For example, on ahot day when interacting with a vending machine that exists in a mixedreality (e.g. having at least a portion of the vending machinerepresented as a virtual object in the mixed reality), a user couldobserve that a virtual entity in the mixed reality, perhaps in a game,also suffers from the heat and requires a drink. Such an approach can beused for vendor promotions or other forms of advertising within a mixedreality.

A step 230 the attributes are used to interact with software that islocated at least partially external to the mobile device. The softwaredetermines an appropriate command set as a function of the attributes orother sensor data for one on or more mixed reality objects that areaccessible by the mobile device. Preferably, software running on acomputer external to the mobile devices uses the attributes to consult adatabase having command set information. It is also contemplated that adatabase of command set information can be located within or accessedfrom a real-world object.

The mixed reality object for which the command set is offered can verysignificantly without departing from the inventive subject matter. Forexample, the mixed reality object can include a virtual object, aportion of the real-world object, or an appliance (e.g. kitchenappliance, consumer electronics, set-top box, TV, or other electronicdevice) as indicated at steps 234, 236, and 238, respectively.Contemplated appliances include those associated with kitchens, businessoffices, consumer electronics, clinics, or other functional devices.

Additionally, at step 231 the local mixed reality object comprises anelement of a game where the game element can also be a real or virtualobject. Obtaining a command set associated with the game elements allowsfor a user of the mobile device to fully interact with a mixed realitygame.

In some gaming embodiments, including at step 234, the real-world objectis a player of the game. When a user captures image data of the player,the user can be offered a set of game related commands that can affectthe player. Contemplated commands include attack, add player, deleteplayer, or other game related commands.

In other embodiments relating to games, as at step 235, the real-worldobject can represent a goal object of the game. Contemplated gameshaving goal objects include virtual tag, scavenger hunts, geo-caching,hide-and-seek, or other games having goal objects. Again, such anapproach provides for promotions or other advertising efforts.

In yet other gaming embodiments, at step 237 the real-world object canhave value within the game system. For example, an image of anadvertisement could result in points added to a player's score or,perhaps, unlock additional commands that can then be used within thegame or with game objects.

Preferred embodiments also optionally allow a user to select a commandfrom the command set at step 250. Contemplated commands include:

At step 252 turning a device on or off by selecting an “on” command oran “off” command

At step 254 changing a channel or station of a TV, radio, or otherappliance.

At step 256 inputting data associated with virtually traveling to a newplace where the place can be a real or virtual location.

At step 258 playing music.

At step 257 inputting data associated with a virtual weapon.

At step 270, the mobile device can optionally include an additionalsensor other than a camera. The additional sensor data acquired by themobile device can also be used to interact with external software to aidin determining a desired command set. At step 271 the sensor dataoptionally provides information associated with a physical location(e.g. latitude or longitude), preferably from a GPS receiver. At step272 the sensor data optionally provides information associated withorientation or acceleration of the mobile device preferably from anaccelerometer, a gyroscope, a compass, or other sensor.

One skilled in the art should now appreciate the numerous advantagesprovided by the disclosed subject matter for interacting with a mixedreality. The disclosed methods provide an easy, intuitive method for aperson to command and control mixed reality objects. Using a cell phoneor PDA offers a ubiquitous interface with which most people familiar.Users simply capture image data of an object and then select a commandfrom an offered command set to control various mixed reality objects orautomatically control the mixed reality objects.

It should be apparent to those skilled in the art that many moremodifications besides those already described are possible withoutdeparting from the inventive concepts herein. The inventive subjectmatter, therefore, is not to be restricted except in the spirit of theappended claims. Moreover, in interpreting both the specification andthe claims, all terms should be interpreted in the broadest possiblemanner consistent with the context. In particular, the terms “comprises”and “comprising” should be interpreted as referring to elements,components, or steps in a non-exclusive manner, indicating that thereferenced elements, components, or steps may be present, or utilized,or combined with other elements, components, or steps that are notexpressly referenced. Where the specification claims refers to at leastone of something selected from the group consisting of A, B, C . . . andN, the text should be interpreted as requiring only one element from thegroup, not A plus N, or B plus N, etc.

1. A method of interacting with a mixed reality, comprising: providing a mobile phone device having a physical handheld display; using the mobile device to electronically capture image data of at least a portion of a multi-part real-world object; using the image data to identify an attribute associated with the real-world object; using the attribute to interact with software being operated at least in part remote to the mobile device to obtain a first command set for a first part of the real-world object and a second, different command set for a second part of the real-world object from a distal database storing a plurality command sets for multiple, mixed reality objects, where the command sets comprise network API addresses corresponding to commands that control the first and the second parts, respectively; offering the first and the second command set to the mobile phone device; combining at least one virtual interface comprising the commands sets with the image data of the real-world object to form mixed reality representations of the first and the second parts; displaying the at least one virtual interface on the handheld display to allow a user to select a command from one of the command sets; and executing the command by issuing call over a network from the mobile device to a corresponding network API address of the command drawn from the network API addresses.
 2. The method of claim 1, wherein the real-world object is an appliance.
 3. The method of claim 1, wherein the first part is physically distinct from the second part.
 4. The method of claim 1, wherein the first part is a component of a consumer electronic appliance.
 5. The method of claim 1, wherein the first part is an element of an electronic game.
 6. The method of claim 5, wherein the attribute indicates the real-world object is a goal object of the game.
 7. The method of claim 1, wherein, the real-world object is a kitchen appliance.
 8. The method of claim 5, wherein the attribute indicates the real-world object has a value in the game.
 9. The method of claim 1, wherein the attribute comprises at least one of the following a name, a class, and an environmental characteristic.
 10. The method of claim 1, wherein the command is one of an “on” command and an “off” command.
 11. The method of claim 1, wherein the selecting of the command results in changing a channel.
 12. The method of claim 1, wherein the selecting of the command results in playing music.
 13. The method of claim 1, wherein the command inputs data associated with a virtual weapon.
 14. The method of claim 1, wherein the command inputs data associated with virtual travel.
 15. The method of claim 1, wherein the image data comprises a still image.
 16. The method of claim 1, wherein the image data comprises a video image.
 17. The method of claim 1, further comprising using an additional sensor associated with the mobile device to interact with the software.
 18. The method of claim 17, wherein the sensor provides information associated with a physical location.
 19. The method of claim 17, wherein the sensor provides information associated with at least one of an orientation and an acceleration of the mobile device. 